Physical IT (information technology) infrastructures are difficult to manage. Changing the network configuration, adding a new machine or storage device are typically difficult manual tasks. This makes such changes expensive and error prone. It also means that the change can take several hours or days to take place, limiting the rate at which reconfiguration can take place to take account of changing business demands. Sometimes the reconfiguration can take months, as more equipment needs to be ordered before it can be implemented.
A physical IT infrastructure can have only one configuration at any one time. Although this configuration might be suitable for some tasks, it is typically sub-optimal for other tasks. For example, an infrastructure designed for running desktop office applications during the day may not be suitable for running complicated numerical analysis applications during the night. In a single physical IT infrastructure, separate tasks can interfere with each other. For example, it has been proposed to use spare compute cycles on desktops and servers to perform large scale computations: grid applications. The problem is how to isolate the network traffic, the data storage and processing of these computations from other tasks using the same infrastructure. Without isolation undesirable interference between the tasks is likely to occur rendering such sharing an unacceptable risk.
In most physical IT infrastructure, resource utilization is very low: 15% is not an uncommon utilization for a server, 5% for a desktop. This means that customers have purchased far more IT infrastructure than they need. HP's UDC (Utility Data Centre) has been applied commercially and addresses some of these problems, by automatic reconfiguration of physical infrastructure: processing machines, network and storage devices. This requires specialized hardware which makes it expensive. In addition in the UDC a physical machine can only ever be in a single physical infrastructure. This means that all programs running on that physical machine will be exposed to the same networking and storage environment: they can interfere with each other and the configuration may not be optimal for all programs. In UDC although a physical machine can be reassigned to different infrastructure instances, called farms, at different times, it can only be assigned to one farm, at any given moment: it is not possible to share a physical machine between farms. This limits the utilization that levels that can be achieved for the hardware, requiring the customer to purchase more hardware than is necessary.
Overlay networks are known and make it easy to change the network configuration, abstracting devices from the configuration of the real network. However, on their own they do not solve the problem of conveniently adding more storage or machines. Nor do they address the problems of the configuration poor utilization of physical infrastructure. Overlay networks are discussed in the literature, for example see: Dynamic Internet Overlay Deployment and Management Using the X-Bone., Joe Touch, Computer Networks, July 2001, pp 117-135;
Violin: Virtual Internetworking on Overlay Infrastructure, X. Jiang, D. Xu, Purdue University Department of Computer Science, CS Technical Report CSD TR 03-027, Purdue University, July 2003.
Storage virtualization is also known. There are many commercial storage virtualization products on the market from HP, IBM, EMC and others. These products are focused on managing the storage available to physical machines and increasing the utilization of storage. Without addition technology they do not solve the problems of how to easily reconfigure the IT infrastructure; the problem of having only one available configuration at any given instance; the problem of low utilization of physical machines.
Virtual machine technology is a known mechanism to run operating system instances on one physical machine independently of other operating system instances. It is known, within a single physical machine, to have two virtual machines connected by a virtual network on this machine. VMware is a known example of virtual machine technology, and can provide isolated environments for different operating system instances running on the same physical machine. However, each operating system instance running under VMware will see the same networking environment, in other words, the same single infrastructure configuration (where infrastructure means arrangement of processing, storage and network resources). This means isolation is insufficient for running grid applications to consume spare compute cycles: the networking can interfere.
The term “virtual” usually means the opposite of real or physical, and is used where there is a level of indirection, or some mediation between the resource user and the physical resource.